Scipion3: A workflow engine for cryo-electron microscopy image processing and structural biology.

Image-processing pipelines require the design of complex workflows combining many different steps that bring the raw acquired data to a final result with biological meaning. In the image-processing domain of cryo-electron microscopy single-particle analysis (cryo-EM SPA), hundreds of steps must be performed to obtain the three-dimensional structure of a biological macromolecule by integrating data spread over thousands of micrographs containing millions of copies of allegedly the same macromolecule. The execution of such complicated workflows demands a specific tool to keep track of all these steps performed. Additionally, due to the extremely low signal-to-noise ratio (SNR), the estimation of any image parameter is heavily affected by noise resulting in a significant fraction of incorrect estimates. Although low SNR and processing millions of images by hundreds of sequential steps requiring substantial computational resources are specific to cryo-EM, these characteristics may be shared by other biological imaging domains. Here, we present Scipion, a Python generic open-source workflow engine specifically adapted for image processing. Its main characteristics are: (a) interoperability, (b) smart object model, (c) gluing operations, (d) comparison operations, (e) wide set of domain-specific operations, (f) execution in streaming, (g) smooth integration in high-performance computing environments, (h) execution with and without graphical capabilities, (i) flexible visualization, (j) user authentication and private access to private data, (k) scripting capabilities, (l) high performance, (m) traceability, (n) reproducibility, (o) self-reporting, (p) reusability, (q) extensibility, (r) software updates, and (s) non-restrictive software licensing.

Image Processing in Cryo-Electron Microscopy of Single Particles: The Power of Combining Methods.

Cryo-electron microscopy has established as a mature structural biology technique to elucidate the three-dimensional structure of biological macromolecules. The Coulomb potential of the sample is imaged by an electron beam, and fast semi-conductor detectors produce movies of the sample under study. These movies have to be further processed by a whole pipeline of image-processing algorithms that produce the final structure of the macromolecule. In this chapter, we illustrate this whole processing pipeline putting in value the strength of "meta algorithms," which are the combination of several algorithms, each one with different mathematical rationale, in order to distinguish correctly from incorrectly estimated parameters. We show how this strategy leads to superior performance of the whole pipeline as well as more confident assessments about the reconstructed structures. The "meta algorithms" strategy is common to many fields and, in particular, it has provided excellent results in bioinformatics. We illustrate this combination using the workflow engine, Scipion.

Continuous flexibility analysis of SARS-CoV-2 spike prefusion structures.

Using a new consensus-based image-processing approach together with principal component analysis, the flexibility and conformational dynamics of the SARS-CoV-2 spike in the prefusion state have been analysed. These studies revealed concerted motions involving the receptor-binding domain (RBD), N-terminal domain, and subdomains 1 and 2 around the previously characterized 1-RBD-up state, which have been modeled as elastic deformations. It is shown that in this data set there are not well defined, stable spike conformations, but virtually a continuum of states. An ensemble map was obtained with minimum bias, from which the extremes of the change along the direction of maximal variance were modeled by flexible fitting. The results provide a warning of the potential image-processing classification instability of these complicated data sets, which has a direct impact on the interpretability of the results.

Continuous flexibility analysis of SARS-CoV-2 Spike prefusion structures.

With the help of novel processing workflows and algorithms, we have obtained a better understanding of the flexibility and conformational dynamics of the SARS-CoV-2 spike in the prefusion state. We have re-analyzed previous cryo-EM data combining 3D clustering approaches with ways to explore a continuous flexibility space based on 3D Principal Component Analysis. These advanced analyses revealed a concerted motion involving the receptor-binding domain (RBD), N-terminal domain (NTD), and subdomain 1 and 2 (SD1 & SD2) around the previously characterized 1-RBD-up state, which have been modeled as elastic deformations. We show that in this dataset there are not well-defined, stable, spike conformations, but virtually a continuum of states moving in a concerted fashion. We obtained an improved resolution ensemble map with minimum bias, from which we model by flexible fitting the extremes of the change along the direction of maximal variance. Moreover, a high-resolution structure of a recently described biochemically stabilized form of the spike is shown to greatly reduce the dynamics observed for the wild-type spike. Our results provide new detailed avenues to potentially restrain the spike dynamics for structure-based drug and vaccine design and at the same time give a warning of the potential image processing classification instability of these complicated datasets, having a direct impact on the interpretability of the results.

Validation of electron microscopy initial models via small angle X-ray scattering curves.

MOTIVATION: Cryo electron microscopy (EM) is currently one of the main tools to reveal the structural information of biological macromolecules. The re-construction of three-dimensional (3D) maps is typically carried out following an iterative process that requires an initial estimation of the 3D map to be refined in subsequent steps. Therefore, its determination is key in the quality of the final results, and there are cases in which it is still an open issue in single particle analysis (SPA). Small angle X-ray scattering (SAXS) is a well-known technique applied to structural biology. It is useful from small nanostructures up to macromolecular ensembles for its ability to obtain low resolution information of the biological sample measuring its X-ray scattering curve. These curves, together with further analysis, are able to yield information on the sizes, shapes and structures of the analyzed particles. RESULTS: In this paper, we show how the low resolution structural information revealed by SAXS is very useful for the validation of EM initial 3D models in SPA, helping the following refinement process to obtain more accurate 3D structures. For this purpose, we approximate the initial map by pseudo-atoms and predict the SAXS curve expected for this pseudo-atomic structure. The match between the predicted and experimental SAXS curves is considered as a good sign of the correctness of the EM initial map. AVAILABILITY AND IMPLEMENTATION: The algorithm is freely available as part of the Scipion 1.2 software at http://scipion.i2pc.es/.

Scipion web tools: Easy to use cryo-EM image processing over the web.

Macromolecular structural determination by Electron Microscopy under cryogenic conditions is revolutionizing the field of structural biology, interesting a large community of potential users. Still, the path from raw images to density maps is complex, and sophisticated image processing suites are required in this process, often demanding the installation and understanding of different software packages. Here, we present Scipion Web Tools, a web-based set of tools/workflows derived from the Scipion image processing framework, specially tailored to nonexpert users in need of very precise answers at several key stages of the structural elucidation process.

ScipionCloud: An integrative and interactive gateway for large scale cryo electron microscopy image processing on commercial and academic clouds.

New instrumentation for cryo electron microscopy (cryoEM) has significantly increased data collection rate as well as data quality, creating bottlenecks at the image processing level. Current image processing model of moving the acquired images from the data source (electron microscope) to desktops or local clusters for processing is encountering many practical limitations. However, computing may also take place in distributed and decentralized environments. In this way, cloud is a new form of accessing computing and storage resources on demand. Here, we evaluate on how this new computational paradigm can be effectively used by extending our current integrative framework for image processing, creating ScipionCloud. This new development has resulted in a full installation of Scipion both in public and private clouds, accessible as public "images", with all the required preinstalled cryoEM software, just requiring a Web browser to access all Graphical User Interfaces. We have profiled the performance of different configurations on Amazon Web Services and the European Federated Cloud, always on architectures incorporating GPU's, and compared them with a local facility. We have also analyzed the economical convenience of different scenarios, so cryoEM scientists have a clearer picture of the setup that is best suited for their needs and budgets.